Blogs

Blogs

For the love of forecasting

2
By

Love can make a person do bad, dangerous, stupid, and irresponsible things.  Love of country can make a politician stray from his wife. Love of Pepsi can make a pop musician lose his hair in a pyrotechnics-gone-bad commercial. Love of acting can make academy award winners accept starring roles in Ishtar. And for me last Wednesday morning, my love of forecasting was so overwhelming and distracting, I forgot to put on shoes and arrived at work in my bedroom slippers.

Beware the "Inside View"

The November 2011 McKinsey Quarterly contains an excerpt from Daniel Kahneman's new book Thinking, Fast and Slow.  Kahneman tells the story of textbook writing project, in which his survey of the co-authors estimated two years (+/- six months) for completion.

Kahneman then asked one of the co-authors if he could think of similar projects, and how long they took to completion.  Here the answer was quite different -- 40% of them failed to finish, and the successful ones had taken seven to 10 years to complete.

At this realization, the more rational course of action would have been to quit.  "None of us was willing to invest six more years in a project with a 40 percent chance of failure." Yet the project continued and the book was completed in eight years -- but was never used.  By then the enthusiasm for the idea had waned.

Kahneman uses this story to illustrate the perils of the "inside view" -- simply extrapolating from our specific circumstances and the information in front of us. This narrow focus on what we know fails to allow for the "unknown unknowns" -- all the crises, unanticipated, and random events that get in the way of our progress. "There are many ways for any plan to fail, and although most of them are too improbable to be anticipated, the likelihood that something will go wrong in a big project is high."

The "outside view" is based on the category or reference class relevant to our prediction.  Here, it was the class of similar book projects.  Kahneman argues that this outside view provides "...a reasonable basis for a baseline prediction: the prediction you make about a case if you know nothing except the category to which it belongs.  This should be the anchor for further adjustments." This estimate can then be adjusted given any case-specific information.

New Product Forecasting by Analogy

Knowingly or not, we employ the outside view when we forecast a new product based on sales of similar products. Such "forecasting by analogy" is perhaps the second most common new product forecasting method (second only to "management forecasting whatever they darn well please").  As Kahneman asserts, "...if the reference class is properly chosen, the outside view will give an indication of where the ballpark is. It may suggest, as it did in our [the book project]case, that the inside-view forecasts are not even close."

Selecting an appropriate reference class is the key step in forecasting by analogy, and is often poorly done. Through mischief or incompetence, we are wont to cherry pick the analogous products (e.g., only the most successful ones) in order to justify the new product forecast we desire.

Find more thorough discussion of this topic in the whitepaper "New Product Forecasting Using Structured Analogies" (available for free download).

Tags Daniel Kahneman Dustin Hoffman Forecasting by Analogy Inside View Ishtar Michael Jackson New Product Forecasting Newt Gingrich Outside View Pepsi Thinking Fast and Slow Warren Beatty
Share

About Author

Mike Gilliland

Product Marketing Manager

Michael Gilliland is a longtime business forecasting practitioner and formerly a Product Marketing Manager for SAS Forecasting. He is on the Board of Directors of the International Institute of Forecasters, and is Associate Editor of their practitioner journal Foresight: The International Journal of Applied Forecasting. Mike is author of The Business Forecasting Deal (Wiley, 2010) and former editor of the free e-book Forecasting with SAS: Special Collection (SAS Press, 2020). He is principal editor of Business Forecasting: Practical Problems and Solutions (Wiley, 2015) and Business Forecasting: The Emerging Role of Artificial Intelligence and Machine Learning (Wiley, 2021). In 2017 Mike received the Institute of Business Forecasting's Lifetime Achievement Award. In 2021 his paper "FVA: A Reality Check on Forecasting Practices" was inducted into the Foresight Hall of Fame. Mike initiated The Business Forecasting Deal blog in 2009 to help expose the seamy underbelly of forecasting practice, and to provide practical solutions to its most vexing problems.

2 Comments

  1. Dave Wehling on

    Very interesting. Are you implying that the MFWTDWP (Management forecasts whatever...) is the inside view?

  2. Mike Gilliland
    Mike Gilliland on

    Hi Dave, I meant to imply that the most common method of new product forecasting is:

    1) Identify the volume/revenue/profit hurdle for getting a new product idea approved.
    2a) If you want to see the product approved, deliver a forecast that meets or exceeds the hurdle.
    2b) If you don't want to see the product approved, deliver a forecast that fails to meet the hurdle.
    3) Justify the forecast by cherry picking past new products that either a) met the hurdle, or b) failed to meet the hurdle (while ignoring past new products that don't fit the story you want to tell).

    This is the "inside view" from an insane asylum.

    I'm not convinced that any new product forecasting method can provide reliably accurate forecasts. I was pleased by Kahneman's warning, "The spectacular accuracy of the outside-view forecast in our specific case was surely a fluke and should not count as evidence for the validity of the outside view." However, I do like the "structured analogy" approach discussed in the white paper -- it can serve as a BS detector when evaluating whatever forecasts that have been concocted.

Back to Top